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TELEVISION PROGRAMMING WITH VARIABLE 
ANCILLARY INFORMATION 



FIELD OF THE INVENTION 

This invention relates generally to the field of interactive television. More 
particularly, this invention relates to a method and apparatus for providing television 
programming with variable ancillary information. 

BACKGROUND OF THE INVENTION 

Television set-top boxes were initially introduced to provide tuning 
capabilities for cable and satellite television systems. While these devices still 
provide that fundamental function, the latest generation of digital set-top boxes now 
often incorporate powerful computers. With such computers available, and with 
the low cost necessitated by the high volume production of such devices, it is now 
possible to expand the usefulness of the television set-top box beyond that of 
merely providing tuning functions for cable and satellite systems. 

The television program "Pop-Up Videos" was recently introduced in which 
music videos are presented with commentary overlaid thereon. Such information 
is presented to the viewer in the form of windows or bubbles that overlay the screen 
image and contain text about, for example, the artist or the music being presented 
to the viewer. This programming is believed to be created by editing a copy of the 
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1 original video content to superimpose the windows or bubbles of information at 

2 appropriate locations. 

3 This popular program is widely enjoyed by viewers wishing to find out 

4 additional information about an artist or musical selection. Moreover, it provides 

5 an avenue to inexpensively reuse entertainment content while providing the viewer 

6 with a new product In many instances, the additional ancillary information overlaid 

7 upon the original video may be of greater interest to the viewer than the original 

8 video, and thus, may attract viewers who are uninterested in the pure video itself. 

9 Unfortunately, while the viewer may be interested in the initial viewing of a video, 
1 0h or the initial viewing of the modified video with pop-up information overlaid thereon, 
1 1*; the viewer is likely to be subjected to either or both of the videos repeatedly. The 
1 2J1 viewer may thus become weary of viewing the same video or ancillary information 
13ji repeatedly. The present invention is intended to address this issue in an invention 
14*; that can be widely adapted to music videos as well as conventional television 
15* programming and pay-per view programming. 

16 S 

1 7(3 SUMMARY OF THE INVENTION 

180 The present invention relates generally to interactive television. Objects, 

19 rr advantages and features of the invention will become apparent to those skilled in 

20 the art upon consideration of the following detailed description of the invention. 

21 In one embodiment of the present invention, a method and apparatus for 

22 presenting variable pop-up windows of ancillary information in television 

23 programming is provided. Multiple instances of ancillary information is transmitted 

24 to a set-top box in association with segments of programming content. The set-top 

25 box selects a different set of ancillary information for display in a pop-up window 

26 whenever the content is presented. In one embodiment, a different set of ancillary 

27 information is presented according to the number of times the programming is 

28 presented. In another embodiment, ancillary information is selected using a 

29 randomizing algorithm. 
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1 A method of presenting entertainment program material consistent with an 

2 embodiment of the invention includes: presenting a first presentation of a segment 

3 of programming having a first window overlaying the segment of programming, the 

4 first window containing a first segment of ancillary information relevant to the 

5 segment of programming; and presenting a second presentation of the segment 

6 of programming having a second window overlaying the segment of programming, 

7 the second window containing a second segment of ancillary information relevant 

8 to the segment of programming. 

9 A method of presenting entertainment program material consistent with 
10 another embodiment of the invention includes the unordered process of: presenting 

a first presentation of a segment of programming having a first window overlaying 

1pj the segment of programming, the first window containing a first segment of 

1$; ancillary information relevant to the segment of programming; and presenting a 

1^ second presentation of the segment of programming having no second window 

1 S overlaying the segment of programming. 

1 fk A method of presenting entertainment program material according to another 

1 fi embodiment includes: presenting a first presentation of a segment of programming 

1 SP having a static window overlaying the segment of programming, the static window 

19 containing a static segment of ancillary information relevant to the segment of 

20 programming; and presenting a second presentation of the segment of 

21 programming having a second window overlaying the segment of programming, the 

22 second window containing a variable segment of ancillary information relevant to 

23 the segment of programming. 

24 A television set-top box consistent with certain embodiments includes a 

25 receiver for receiving signals representing segments of programming and signals 

26 representing a plurality of segments of ancillary information, and delivering the 

27 signals representing segments of programming to a display interface. A central 

28 processor is programmed to carry out a process of delivering a first presentation 

29 of a segment of programming having a first window overlaying the segment of 

30 programming to the display interface, the first window containing a first segment 
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1 of ancillary information relevant to the segment of programming; and delivering a 

2 second presentation of the segment of programming having a second window 

3 overlaying the segment of programming to the display interface, the second 

4 window containing a second segment of ancillary information relevant to the 

5 segment of programming. 

6 A television set-top box consistent with another embodiment of the invention 

7 includes a receiver for receiving signals representing segments of programming 

8 and signals representing a plurality of segments of ancillary information, and 

9 delivering the signals representing segments of programming to a display interface. 
1(m A central processor is programmed to deliver a first presentation of a segment of 
1 p programming having a first window overlaying the segment of programming to the 
12r! display interface, the first window containing a first segment of ancillary information 
1 3| relevant to the segment of programming; and deliver a second presentation of the 
14^ segment of programming having no second window overlaying the segment of 
1 5 programming to the display interface. 

1$r In yet another embodiment of a television set-top box consistent with the 

17Q invention, a receiver for receives signals representing segments of programming 

18q and signals representing a plurality of segments of ancillary information, and 

1 Jf : delivering the signals representing segments of programming to a display interface. 

20 A central processor is programmed to deliver a first presentation of a segment of 

21 programming having a static window overlaying the segment of programming to the 

22 display interface, the static window containing a static segment of ancillary 

23 information relevant to the segment of programming; and deliver a second 

24 presentation of the segment of programming having a second window overlaying 

25 the segment of programming to the display interface, the second window 

26 containing a variable segment of ancillary information relevant to the segment of 

27 programming. 

28 In another embodiment, a storage medium storing instructions which, when 

29 executed on a programmed processor, carry out a method of presenting 

30 entertainment program material including presenting a first presentation of a 
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1 segment of programming having a first window overlaying the segment of 

2 programming, the first window containing a first segment of ancillary information 

3 relevant to the segment of programming; and presenting a second presentation of 

4 the segment of programming having a second window overlaying the segment of 

5 programming, the second window containing a second segment of ancillary 

6 information relevant to the segment of programming. 

7 In another embodiment, a storage medium storing instructions which, when 

8 executed on a programmed processor, carry out a method of presenting 

9 entertainment program material, including the unordered process of: presenting a 
1 jjj first presentation of a segment of programming having a first window overlaying the 
1 fi segment of programming, the first window containing a first segment of ancillary 
121 information relevant to the segment of programming; and presenting a second 
1 h presentation of the segment of programming having no second window overlaying 
1|f the segment of programming. 

1 ? In another embodiment, a storage medium storing instructions which, when 

16;, executed on a programmed processor, carry out a method of presenting 

Ifi entertainment program material including presenting a first presentation of a 

18j segment of programming having a static window overlaying the segment of 

19 programming, the static window containing a static segment of ancillary 

20 information relevant to the segment of programming; and presenting a second 

21 presentation of the segment of programming having a second window overlaying 

22 the segment of programming, the second window containing a variable segment 

23 of ancillary information relevant to the segment of programming. 

24 The above summaries are intended to illustrate exemplary embodiments of 

25 the invention, which will be best understood in conjunction with the detailed 

26 description to follow, and are not intended to limit the scope of the appended 

27 claims. 
28 
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1 BRIEF DESCRIPTION OF THE DRAWINGS 

2 The features of the invention believed to be novel are set forth with 

3 particularity in the appended claims. The invention itself however, both as to 

4 organization and method of operation, together with objects and advantages 

5 thereof, may be best understood by reference to the following detailed description 

6 of the invention, which describes certain exemplary embodiments of the invention, 

7 taken in conjunction with the accompanying drawings in which: 

8 FIGURE 1 is a system block diagram of a system using a set-top box, 

9 FIGURE 2 is a functional block diagram of a digital set-top box suitable for 
1 Q:;| use with the present invention. 

1 tj FIGURE 3 depicts an exemplary screen image with a first pop-up window 

1^5 containing ancillary information. 

135 FIGURE 4 depicts an exemplary screen image with a second pop-up 

l4i window containing ancillary information. 

1£L ; FIGURE 5 depicts an exemplary screen image with a third pop-up window 

1 6f containing ancillary information. 

17bj FIGURE 6 depicts an exemplary screen image with a fourth pop-up window 

1 8h* containing ancillary information. 

19 FIGURE 7 is a flow chart describing a first embodiment of the present 

20 invention. 

21 FIGURE 8 is a flow chart describing a second embodiment of the present 

22 invention. 
23 

24 DETAILED DESCRIPTION OF THE INVENTION 

25 

26 While this invention is susceptible of embodiment in many different forms, 

27 there is shown in the drawings and will herein be described in detail specific 

28 embodiments, with the understanding that the present disclosure is to be 

29 considered as an example of the principles of the invention and not intended to limit 
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1 the invention to the specific embodiments shown and described. In the description 

2 below, like reference numerals are used to describe the same, similar or 

3 corresponding parts in the several views of the drawings. 

4 Referring to FIGURE 1 , a block diagram for an exemplary interactive cable 

5 or satellite television (TV) system 100 is shown. The system 100 includes, at a 

6 head end of the service provider 10, a media server 12 for providing, on demand, 

7 movies and other programming obtained from a media database 14. The media 

8 server 12 might also provide additional content such as interviews with the actors, 

9 games, advertisements, available merchandise, associated Web pages, interactive 
1QL games and other related content. The system 100 also includes an electronic 
1fl programming guide (EPG) server 16 and a program listing database 18 for 
1@1 generating an EPG. Set-top box 22 can generally provide for bidirectional 
13b; communication over a transmission medium 20 in the case of a cable STB 22. In 
14J other embodiments, bidirectional communication can be effected using 
1| asymmetrical communication techniques possibly using dual communication 
1 §: media - - one for the uplink and one for the downlink. In any event, the STB 22 can 
1X3 have its own Universal Resource Locator (URL) or IP address or other unique 
1 §3 identifier assigned thereto to provide for addressability by the head end and users 
1 in of the Internet. 

20 The media server 12 and EPG server 16 are operatively coupled by 

21 transmission medium 20 to a set-top box (STB) 22. The transmission medium 20 

22 may include, for example, a conventional coaxial cable network, a fiber optic cable 

23 network, telephone system, twisted pair, a satellite communication system, a radio 

24 frequency (RF) system, a microwave system, other wireless systems, a 

25 combination of wired and wireless systems or any of a variety of known electronic 

26 transmission mediums. In the case of a cable television network, transmission 

27 medium 20 is commonly realized at the subscriber's premises as a coaxial cable 

28 that is connected to a suitable cable connector at the rear panel of the STB 22. In 

29 the case of a Direct Satellite System (DSS), the STB 22 is often referred to as an 

30 Integrated Receiver Decoder (IRD). In the case of a DSS system, the transmission 
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1 medium is a satellite transmission at an appropriate microwave band. Such 

2 transmissions are typically received by a satellite dish antenna with an integral Low 

3 Noise Block (LNB) that serves as a down-converter to convert the signal to a lower 

4 frequency for processing by the STB 22. 

5 The exemplary system 100 further includes a TV 24, such as a digital 

6 television, having a display 26 for displaying programming, an EPG, etc. The STB 

7 22 may be coupled to the TV 24 and various other audio/visual devices 26 (such as 

8 audio systems, Personal Video Recorders (PVRs), Video Tape Recorders (VTRs), 

9 Video Cassette Recorders (VCRs) and the like), storage devices (e.g., hard disc 
10 s drives) and Internet Appliances 28 (such as email devices, home appliances, 
1 f{ storage devices, network devices, and other Internet Enabled Appliances) by an 
1211 appropriate interface 30, which can be any suitable analog or digital interface. In 
13g one embodiment, interface 30 conforms to an interface standard such as the 
14ff Institute of Electrical and Electronics Engineers (IEEE) 1394 standard, but could 
15 also be wholly or partially supported by a DVI interface (Digital Visual Interface - 
1C- Digital Display Working Group, www.ddwg.org) or other suitable interface. 

1 7^ The STB 22 may include a central processing unit (CPU) such as a 

1 8D microprocessor and memory such as Random Access Memory (RAM), Read Only 

1 9" Memory (ROM), flash memory, mass storage such as a hard disc drive, floppy disc 

20 drive, optical disc drive or may accommodate other electronic storage media, etc. 

21 Such memory and storage media is suitable for storing data as well as instructions 

22 for programmed processes for execution on the CPU, as will be discussed later. 

23 Information and programs stored on the electronic storage media or memory may 

24 also be transported over any suitable transmission medium such as that illustrated 

25 as 20. STB 22 may include circuitry suitable for audio decoding and processing, 

26 the decoding of video data compressed in accordance with a compression 

27 standard such as the Motion Pictures Experts Group (MPEG) standard and other 

28 processing to form a controller or central hub. Alternatively, components of the 

29 STB 22 may be incorporated into the TV 24 itself, thus eliminating the STB 22. 
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1 Further, a computer having a tuner device and modem may be equivalently 

2 substituted for the TV 24 and STB 22. 

3 By way of example, the STB 22 may be coupled to devices such as a 

4 personal computer, video cassette recorder, camcorder, digital camera, personal 

5 digital assistant and other audio/visual or Internet related devices. In addition, a 

6 data transport architecture, such as that set forth by an industry group which 

7 includes Sony Corporation and known as the Home Audio-Video Interoperability 

8 (HAVi) architecture may be utilized to enable interoperability among devices on a 

9 network regardless of the manufacturer of the device. This forms a home network 
1QL. system wherein electronic devices and Internet appliances are compatible with 
10 each other. The STB 22 runs an operating system suitable for a home network 
1|l system such as Sony Corporation's Aperios™ real time operating system. Other 
1S; operating systems could also be used. 

1 &J The STB 22 includes an infrared (IR) receiver 34 for receiving IR signals from 

1 fT an input device such as remote control 36. Alternatively, it is noted that many other 

lf J control communication methods may be utilized besides IR, such as wired or 

ID wireless radio frequency, etc. In addition, it can be readily appreciated that the 

l|i input device 36 may be any device suitable for controlling the STB 22 such as a 

19" remote control, personal digital assistant, laptop computer, keyboard or computer 

20 mouse. In addition, an input device in the form of a control panel located on the TV 

21 24 or the STB 22 can be provided. 

22 The STB 22 may also be coupled to an independent service provider (ISP) 

23 host 38 by a suitable connection including dial-up connections, DSL (Digital 

24 Subscriber Line) or the same transmission medium 20 described above (e.g., using 

25 a cable modem) to, thus, provide access to services and content from the ISP and 

26 the Internet. The ISP host 38 provides various content to the user that is obtained 

27 from a content database 42. STB 22 may also be used as an Internet access 

28 device to obtain information and content from remote servers such as remote 

29 server 48 via the Internet 44 using host 38 operating as an Internet portal, for 

30 example. In certain satellite STB environments, the data can be downloaded at 
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very high speed from a satellite link, with asymmetrical upload speed from the set- 
top box provided via a dial-up or DSL connection. 

While the arrangement illustrated in FIGURE 1 shows a plurality of servers 
and databases depicted as independent devices, any one or more of the servers 
can operate as server software residing on a single computer. Moreover, although 
not explicitly illustrated, the servers may operate in a coordinated manner under 
centralized or distributed control to provide multiple services as a Multiple Service 
Operator (MSO) in a known manner. Additionally, the services provided by the 
servers shown in FIGURE 1 may actually reside in other locations, but from the 
perspective of the user of STB 22, the service provider 1 0 serves as a portal to the 
services shown. Those skilled in the art will appreciate that the illustration of 
FIGURE 1 represents a simplified depiction of a cable system configuration shown 
simply as service provider 10. The actual configuration of the service provider's 
equipment is more likely to follow a configuration defined by the CableLabs 
OpenCable™ specification. The simplified illustration shown is intended to simplify 
the discussion of the service provider 10's operation without unnecessarily 
burdening the discussion with architectural details that will be evident to those 
skilled in the art. Those details can be found in the publicly available CableLabs 
OpenCable™ specification or in the text "OpenCable Architecture (Fundamentals)" 
by Michael Adams, Cisco Press, Nov. 1999. 

Referring now to FIGURE 2, a typical system configuration for a digital set- 
top box 22 is illustrated. In this exemplary set-top box, the transmission medium 
20, such as a coaxial cable, is coupled by a suitable interface through a diplexer 
1 02 to a tuner 1 04. Tuner 1 04 may, for example, include a broadcast in-band tuner 
for receiving content, an out-of-band (OOB) tuner for receiving data transmissions. 
A return path through diplexer 1 02 provides an OOB return path for outbound data 
(destined for example for the head end). A separate tuner (not shown) may be 
provided to receive conventional RF broadcast television channels. Modulated 
information formatted, for example, as MPEG-2 information is then demodulated 
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1 at a demodulator 106. The demodulated information at the output of demodulator 

2 106 is provided to a demultiplexer and descrambler circuit 110 where the 

3 information is separated into discrete channels of programming. The programming 

4 is divided into packets, each packet bearing an identifier called a Packet ID (PID) 

5 that identifies the packet as containing a particular type of data (e.g., audio, video, 

6 data). The demodulator and descrambler circuit 110 also decrypts encrypted 

7 information in accordance with a decryption algorithm to prevent unauthorized 

8 access to programming content, for example. 

9 Audio packets from the demultiplexer 110 (those identified with an audio 
1Q ; PID) are decrypted and forwarded to an audio decoder 1 14 where they may be 
ill converted to analog audio to drive a speaker system (e.g., stereo or home theater 
1S multiple channel audio systems) or other audio system 116 (e.g., stereo or home 
1|! theater multiple channel amplifier and speaker systems) or may simply provide 
1&j decoded audio out at 118. Video packets from the demultiplexer 110 (those 
15" identified with a video PID) are decrypted and forwarded to a video decoder 122. 
1fP In a similar manner, data packets from the demultiplexer 1 1 0 (those identified with 
lib a data PID) are decrypted and forwarded to a data decoder 126. 

1 Decoded data packets from data decoder 1 26 are sent to the set-top box's 

1 computer system via the system bus 1 30. A central processing unit (CPU) 1 32 can 

20 thus access the decoded data from data decoder 126 via the system bus 130. 

21 Video data decoded by video decoder 122 is passed to a graphics processor 1 36, 

22 which is a computer optimized to processes graphics information rapidly. Graphics 

23 processor 136 is particularly useful in processing graphics intensive data 

24 associated with Internet browsing, gaming and multimedia applications such as 

25 those associated with MHEG (Multimedia and Hypermedia information coding 

26 Experts Group) set-top box applications. It should be noted, however, that the 

27 function of graphics processor 136 may be unnecessary in some set-top box 

28 designs having lower capabilities, and the function of the graphics processor 136 

29 may be handled by the CPU 1 32 in some applications where the decoded video is 

30 passed directly from the demultiplexer 1 1 0 to a video encoder. Graphics processor 
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1 1 36 is also coupled to the system bus 1 30 and operates under the control of CPU 

2 132. 

3 Many set-top boxes such as STB 22 may incorporate a smart card reader 

4 1 40 for communicating with a so called "smart card," often serving as a Conditional 

5 Access Module (CAM). The CAM typically includes a central processor unit (CPU) 

6 of its own along with associated RAM and ROM memory. Smart card reader 140 

7 is used to couple the system bus of STB 22 to the smart card serving as a CAM 

8 (not shown). Such smart card based CAMs are conventionally utilized for 

9 authentication of the user and authentication of transactions carried out by the user 
as well as authorization of services and storage of authorized cryptography keys. 

z f\ For example, the CAM can be used to provide the key for decoding incoming 

d;2 cryptographic data for content that the CAM determines the user is authorized to 

*%3 receive. 

£'?4 STB 22 can operate in a bidirectional communication mode so that data and 

;15 other information can be transmitted not only from the system's head end to the 

LI 6 end user, or from a service provider to the end user of the STB 22, but also, from 

U 7 the end user upstream using an out-of-band channel. In one embodiment, such 

r 18 data passes through the system bus 130 to a modulator 144 through the diplexer 

19 102 and out through the transmission medium 20. This capability is used to 

20 provide a mechanism for the STB 22 and/or its user to send information to the head 

21 end (e.g., service requests or changes, registration information, etc.) as well as to 

22 provide fast outbound communication with the Internet or other services provided 

23 at the head end to the end user. 

24 Set-top box 22 may include any of a plurality of I/O (Input/Output) interfaces 

25 represented by I/O interfaces 146 that permit interconnection of I/O devices to the 

26 set-top box 22. By way of example, and not limitation, a serial RS-232 port 1 50 can 

27 be provided to enable interconnection to any suitable serial device supported by the 

28 STB 22's internal software. Similarly, communication with appropriately compatible 

29 devices can be provided via an Ethernet port 1 52, a USB (Universal Serial Bus) port 

30 154, an IEEE 1394 (so-called firewire™ or Mink™) or IEEE 1394 wide port 156, S- 

Docket No.: SNY-P4151 -12- PATENT 



1 video port 1 58 or infrared port 160. Such interfaces can be utilized to interconnect 

2 the STB 22 with any of a variety of accessory devices such as storage devices, 

3 audio / visual devices 26, gaming devices (not shown), Internet Appliances 28, etc. 

4 I/O interfaces 146 can include a modem (be it dial-up, cable, DSL or other 

5 technology modem) having a modem port 1 62 to facilitate high speed or alternative 

6 access to the Internet or other data communication functions. In one preferred 

7 embodiment, modem port 162 is that of a DOCSIS (Data Over Cable System 

8 Interface Specification) cable modem to facilitate high speed network access over 

9 a cable system, and port 162 is appropriately coupled to the transmission medium 
J|0 20 embodied as a coaxial cable. Thus, the STB 22 can carry out bidirectional 
J.1 communication via the DOCSIS cable modem with the STB 22 being identified by 
ii2 a unique IP address. The DOCSIS specification is publically available. 

l|3 A PS/2 or other keyboard / mouse / joystick interface such as 164 can be 

H4 provided to permit ease of data entry to the STB 22. Such inputs provide the user 

Cil 5 with the ability to easily enter data and/or navigate using pointing devices. Pointing 

f;|6 devices such as a mouse or joystick may be used in gaming applications, 

^ 7 Of course, STB 22 also may incorporate basic video outputs 1 66 that can be 

N8 used for direct connection to a television set such as 24 instead of (or in addition 

19 to) an IEEE 1394 connection such as that illustrated as 30. In one embodiment, 

20 Video output 166 can provide composite video formatted as NTSC (National 

21 Television System Committee) video. In some embodiments, the video output 1 66 

22 can be provided by a direct connection to the graphics processor 136 or the 

23 demultiplexer / descrambler 110 rather than passing through the system bus 130 

24 as illustrated in the exemplary block diagram. S-Video signals from output 1 58 can 

25 be similarly provided without passing through the system bus 1 30 if desired in other 

26 embodiments. 

27 The infrared port 160 can be embodied as an infrared receiver 34 as 

28 illustrated in FIGURE 1, to receive commands from an infrared remote control 36, 

29 infrared keyboard or other infrared control device. Although not explicitly shown, 

30 front panel controls may be used in some embodiments to directly control the 
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1 operation of the STB 22 through a front panel control interface as one of interfaces 

2 146. Selected interfaces such as those described above and others can be 

3 provided in STB 22 in various combinations as required or desired. 

4 STB 22 will more commonly, as time goes on, include a disc drive interface 

5 1 70 and disc drive mass storage 1 72 for user storage of content and data as well 

6 as providing storage of programs operating on CPU 1 32. STB 22 may also include 

7 floppy disc drives, CD ROM drives, CD R/W drives, DVD drives, etc. CPU 132, in 

8 order to operate as a computer, is coupled through the system bus 1 30 (or through 

9 a multiple bus architecture) to memory 176. Memory 178 may include a 
tp combination any suitable memory technology including Random Access Memory 
1i (RAM), Read Only Memory (ROM), Flash memory, Electrically Erasable 
|i Programmable Read Only Memory (EEPROM), etc. 

f3 While the above exemplary system including STB 22 is illustrative of the 

$4 basic components of a digital set-top box suitable for use with the present 

tf> invention, the architecture shown should not be considered limiting since many 

|& variations of the hardware configuration are possible without departing from the 

W present invention. The present invention could, for example, also be implemented 

ife in more advanced architectures such as that disclosed in U.S. Patent Application 

19 Serial No. 09/473,625, filed Dec. 29, 1999, Docket No. SONY-50N3508 entitled 

20 "Improved Internet Set-Top Box Having and In-Band Tuner and Cable Modem" to 

21 Jun Maruo and Atsushi Kagami. This application describes a set-top box using a 

22 multiple bus architecture with a high level of encryption between components for 

23 added security. This application is hereby incorporated by reference as though 

24 disclosed fully herein. 

25 In general, during operation of the STB 22, an appropriate operating 

26 systeml 80 such as, for example, Sony Corporation's Aperios™ real time operating 

27 system is loaded into, or is permanently stored in, active memory along with the 

28 appropriate drivers for communication with the various interfaces. In other 

29 embodiments, other operating systems such as Microsoft Corporation's Windows 

30 CE™ could be used without departing from the present invention. Along with the 
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1 operating system and associated drivers, the STB 22 usually operates using 

2 browser software 182 in active memory or may permanently reside in ROM, 

3 EEPROM or Flash memory, for example. The browser software 182 typically 

4 operates as the mechanism for viewing not only web pages on the Internet, but 

5 also serves as the mechanism for viewing an Electronic Program Guide (EPG) 

6 formatted as an HTML document. The browser 182 can also provide the 

7 mechanism for viewing normal programming (wherein normal programming is 

8 viewed as an HTML video window - often occupying the entire area of screen 26). 

9 STB software architectures vary depending upon the operating system. 
3i0 However, in general, all such architectures generally include, at the lowest layer, 
,|l various hardware interface layers. Next is an operating system layer as previously 
m$2 described. The software architectures of modern STB have generally evolved to 
i!j3 include a next layer referred to as "middleware." Such middleware permits 
Ha applications to run on multiple platforms with little regard for the actual operating 
£-35 system in place. Middleware standards are still evolving at this writing, but are 
;16 commonly based upon Javascript and HTML (hypertext Markup Language) virtual 

7 machines. At the top layer is the application layer where user applications and the 

H8 like reside (e.g., browsing, email, EPG, Video On Demand (VOD), rich multimedia 

19 applications, pay per view, etc.). The current invention can be utilized with any 

20 suitable set-top box software and hardware architecture. 

21 In accordance with the present invention, a pop-up window is used to carry 

22 ancillary information associated with television programming. This ancillary 

23 information can be encoded within the television signal in any suitable manner. For 

24 example, the information can be encoded within the vertical or horizontal blanking 

25 intervals in one embodiment, and in another embodiment, the information can be 

26 encoded using any of the current or emerging standards for embedding information 

27 into a digital television information stream without limitation. The terms "window" 

28 and "pop-up window" as used herein are intended to embrace areas of the 

29 television screen which are overlaid with rectangles, bubbles, ovals or other shapes 

30 containing textual or graphical information. 

Docket No.: SNY-P4151 -1 5- PATENT 



1 In various advanced television proposals, such as the ATVEF (Advanced 

2 Television Enhancement Forum) proposal, it is possible to embed information 

3 such as Internet addresses and other ancillary information within a television 

4 signal. Such ancillary information can be decoded at the television set and used 

5 for various purposes. Additionally, a technique for providing such information is 

6 now available and in use by Wink Communications of Alameda, CA to embed 

7 ancillary information into "Wink™ Enabled" television programming. Techniques 

8 such as these can be utilized to provide the viewer's set-top box with the ancillary 

9 information used in accordance with the present invention. 

"CD In accordance with the present invention, the viewer is presented with pop- 

Vl up windows during viewing of a television program. The ancillary information 

112 contained within the pop-up windows changes in any of a plurality of ways 

[13 depending upon the algorithm selected for implementation. Those skilled in the art 

' C 4A will recognize many possible algorithms that are suitable for implementing the 

;1;5 present invention, with the constraint that the user is presented with varying content 

;46 within pop-up windows whenever the program is viewed. The term program is 

47 used loosely herein to represent any television content such as a conventional 

P 1 8 program, movie, pay-per-view event or even segments of a program (e.g. as in 

19 music videos making up a program of a collection of such music videos). 

20 An example of the present invention is illustrated in the simulated screen 

21 images of FIGURES 3-6. In each of these images, the program content (in this 

22 example a music video portraying a jazz saxophone player) remains the same. But, 

23 each time the video is viewed, the pop-up window displays a different segment of 

24 ancillary information. In FIGURE 3, the screen image 300 portrays the performer 

25 at a particular time in the program with pop-up window 31 0 providing a segment of 

26 ancillary information regarding his sequence of hits. In FIGURE 4, the screen 

27 image 400 portrays the performer at the same particular time in the program with 

28 pop-up window 410 providing a segment of ancillary information regarding his 

29 experience in playing the saxophone. In FIGURE 5, the screen image 500 portrays 
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1 the performer at the same particular time in the program with pop-up window 510 

2 providing a segment of ancillary information regarding his other musical skills 

3 besides saxophone. Finally, in FIGURE 6, the screen image 600 portrays the 

4 performer, again at the same particular time in the program, with pop-up window 

5 610 providing a segment of ancillary information regarding his wife's musical 

6 career. While four examples are shown, this should not be considered limiting. 

7 As illustrated, the same segment of video can be used to trigger a different 

8 set of ancillary information to be displayed to a viewer on different viewings of the 
r , 9 program material. In this example, not only is the ancillary information appearing 
^1 0 in the window varied, but the location of the window and shape of the window is 
yf 1 also varied to further enhance the interest level. In other embodiments, the shape, 
^12 size and location of the window can be static or can be varied aiong with the 
y 3 varying of the ancillary information, as desired in implementation. In this example 
s 14 also, the ancillary information always appears at the same location in the program, 
; i5 but this too is not to be limiting. Information can be randomly or systematically 
-j16 displayed in a manner that varies from viewing to viewing without departing from 
517 the invention. 

■ : 18 In order to accomplish this, the ancillary information in the preferred 

19 embodiment is transmitted to the set-top box using any suitable technique and 

20 stored in local storage (e.g. the disc drive 1 72 or memory 1 76) for later retrieval and 

21 merging with the programming content during playback. This transmission can be 

22 carried out prior to the programming or during the programming. In accordance 

23 with one embodiment, when a marker is reached in the video signal, the set-top 

24 box processor 132, under program control, places the ancillary information in a 

25 window that it then overlays on the program information by superimposing the 

26 window over the video information in a known manner and delivers the combined 

27 information (programming segment plus window containing ancillary information) 

28 it to the display interface (e.g. 156 or 166). (In other embodiments, the ancillary 

29 information may include insertion times or frames for the insertion of the window.) 
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1 The marker may include various information used by the programmed 

2 processor 1 32 to insert the proper pop-up window on the screen. For example, the 

3 marker may include a popup location identifier that uniquely identifies a location in 

4 the program so that the marker can be matched with a suitable pop-up window of 

5 ancillary information. The marker may also determine the size and location of the 

6 pop-up window, or this can be otherwise determined. 

7 Upon reaching a pop-up marker in the video stream, the programmed 

8 processor 1 32 retrieves a suitable segment of ancillary information for insertion into 

9 the video stream. In accordance with one illustrative embodiment, the programmed 
ft) processor looks up the already received ancillary information in a table stored on 
M the disc drive 172. The table may contain information similar to that of TABLE 1. 
'42 TABLE 1 below illustrates a portion of the table containing information relating to 
\1B the pop-up windows illustrated in FIGURES 3-6 shown previously. In this example, 
/14 the pop-up window is located in location number 8 in the programming segment. 
H ; 5 A plurality of segments of ancillary information is illustrated including segments 1 , 
□6 2, 3 and 4 corresponding to 31 0, 41 0 r 51 0 and 51 0 of FIGURES 3-6. Additionally, 
d7 in this embodiment, TABLE 1 includes a time duration for display of the ancillary 

18 information. In other embodiments, this can be represented by an ending frame 

19 number. In still other embodiments, a video marker is not used at all. Instead, 

20 TABLE 1 would contain a starting frame number and an ending frame number or 

21 a starting run time and an ending run time that would determine when the ancillary 

22 information is presented to the viewer, as will be described later. 



23 
24 
25 
26 
27 
28 
29 
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^ 6 0 seconds 
4 7.5 seconds 


BOB HAS HAD FOUR 
CONSECUTIVE NUMBER ONE 
HITS ON THE JAZZ CHARTS 
SINCE HIS DEBUT IN 1998 

BOB HAD ONLY BEEN PLAYING 
SAX FOR THREE YEARS WHEN 
HE SIGNED HIS FIRST RECORD 
DEAL! 

BOB ALSO PLAYS CLARINET, 
CELLO, PIANO AND 
HARMONICA 

BOB'S WIFE JULIE PLAYS 

* u*-\t iki r\Ki NIC TLJIDP^ Al Rl IIWI 

VIOLIN ON Hlo 1 HIKU ALDUivi 
AN IS NOW TOURING WITH HIM 
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TABLE 1 



The above information presented in TABLE 1, is intended to be illustrative 
of the type of information transmitted as the ancillary data and should not be 
considered limiting. The ancillary data could also include information determining 
the location and shape of the window, starting frame number, ending frame number 
and/or other information without departing from the invention. 
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1 Referring now to FIGURE 7, an embodiment of the present invention is 

2 illustrated as process 700 starting at 704. At 708, the programming is received 

3 including embedded ancillary information. At this point, it is assumed that the 

4 needed ancillary information has been received by the set-top box 22 in advance 

5 of the time it should be displayed. At 71 2, the programming content is started and 

6 played until the appropriate time for a pop-up window containing an ancillary 

7 information segment is reached at 718. This can be determined by receiving a 

8 marker in the segment of programming as illustrated in TABLE 1 (or alternatively 

9 by reaching a frame number or run time corresponding to an appropriate time for 
$0 the pop-up window to appear). When the appropriate time is reached for a pop-up 
,11 window to be displayed at 71 8, the programmed processor randomly selects from 

the available pop-up ancillary information segments available for presentation at 

[1;3 this time at 724 (e.g. any of the segments associated with a given pop-up location 

M?4 identifier or pop-up start time or frame). This ancillary information is then overlaid 

05 on the programming segment at 730 for a predetermined period of time. This 

r16 process repeats until the end of the programming segment is reached at 734 and 

;j 7 the process ends at 740. 

HI 8 Referring now to FIGURE 8 in conjunction with TABLE 2 below, another 

19 embodiment of the present invention is illustrated as process 800 starting at 804. 

20 In this embodiment, the ancillary information is arranged in sets of data, but those 

21 skilled in the art will appreciate that TABLE 2 contains equivalent information to 

22 that of TABLE 1 except it is presented in a different arrangement for ease of 

23 understanding and to illustrate another technique for starting and stopping the pop- 

24 up window. At 808, the receipt of the program content at the set-top box starts 

25 including information such as that in TABLE 2 representing multiple sets of 

26 ancillary data (set 1, set 2, ... etc.). In this example, the set-top box records the 

27 number of times a particular programming segment has been previously viewed. 

28 This can be tracked by creating an identifier for each relevant programming 

29 segment and storing this information in a database in disc drive 1 72, for example. 
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1 At 812, the set-top box determines the nature of the programming segment being 

2 viewed and determines how many times the segment has been viewed previously 

3 (N). The programming commences play at 818 until time for a pop-up window of 

4 information as determined, in this example, by a frame number for a digital 

5 television image being displayed at 822. By way of example, if frame number 

6 123,456 has been reached and this is the 5 th time this programming segment has 

7 been viewed (i.e. N=5), the pop-up window 510 of FIGURE 5 is displayed in 

8 accordance with the entry points of TABLE 2 defined by the frame number and N. 

9 Thus, at 828, the appropriate ancillary information segment is selected for 
%0 presentation in the pop-up window at 832 until the end frame number (in this 
,1:1 example, 123,636) is reached. This process continues until the end of the 
■A2 programming is reached at 836. At this point, the counter N is incremented in the 
;Vl3 database by the programmed processor 1 32 at 840 and the process ends at 844. 
H4 Of course those skilled in the art will appreciate that many variations are possible 
□ 5 without departing from the invention. For example, the counter can be incremented 
*J6 at another location or the start and stop time can be varied. Moreover, the size, 
j r 3 7 shape and location of the pop-up window can be varied (either by reference to data 
14 8 in the table, or by random creation of the window by processor 1 32 or server 1 2, for 

1 9 example) without departing from the invention. 

20 Note that one variation of the present embodiment is illustrated at frame 

21 123,456 for set number 6. In this variation, which could be represented in many 

22 ways including a total lack of information associated with frame 123,456, no 

23 ancillary information is provided. In this manner, not only can the content of the 

24 pop-up window be varied, the windows can apparently appear at different places 

25 in each presentation of the segment of programming. That is, for example, a given 

26 segment of programming might include 30 pop-up windows. But, 90 pop-up 

27 window locations can be defined with only 30 used in any given presentation of the 

28 segment of programming with hundreds of segments of ancillary information stored 

29 and available for viewing. In this manner, not only is the viewer presented with 
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1 different ancillary information each time the segment of programming is viewed, but 

2 additionally, the pop-up windows appear in unpredictable locations each time. 

3 While the processes of FIGURES 7 and 8 have been described in terms of 

4 a system in which the ancillary information segments are selected for display at 

5 STB 22 using programmed processor 132, this should not be considered limiting. 

6 In another embodiment, either process, or similar processes, can be carried out at 

7 the service provider or other program broadcaster. In such embodiments, for < 

8 example, media server 12 of service provider 1 0 includes the ancillary information 
: J9 along with the programming content in database 14. A programmed processor 
lb within media server 1 2 then merges the ancillary information into appropriate pop- 
rfjt up windows in accordance with any suitable random or systematic algorithm, and 
^2 the merged information is then broadcast to the subscribers' STBs. The process 
!1S can be carried out as a batch job or dynamically as the broadcast occurs. In the 
J4 case of the process using multiple sets of ancillary data, a new set can be selected 
H ; 5 for each broadcast rather than for each reception as previously described. Many 
□6 variations will occur to those skilled in the art without departing from the invention. 
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6 123,456 123,456 








7 123,456 123,681 


RfjD'o WIFE JULIE PLAYS 
VIOLIN ON HIS THIRD ALBUM 
AN IS NOW TOURING WITH HIM 







TABLE 2 



The marker based system discussed previously is illustrated in the diagram 
of FIGURE 9. In this process, a stream of video contains video portions 904 and 
pop-up window markers 910. Upon encountering a marker at 910, the 

O o PATENT 
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1 programmed processor 132 or media server 12 selects a segment of ancillary 

2 information at 920 using any appropriate algorithm suitable for varying the content 

3 of the pop-up windows for the viewer. The processor then modifies the video 

4 stream by inserting the pop-up window to produce the modified stream at 930 

5 containing the video signal with the window overlaid thereon. Marker 91 0 may be 

6 removed from the resultant video stream in some embodiments. This video stream 

7 930 can then be processed and viewed in the normal manner. 

8 In another variation of the invention, the sequence of programming can 
,J9 include default pop-up windows that are static (i.e. remain the same on each 
^J0 presentation of the segment of programmed content). The variable ancillary 
Jf 1 information can then be overlaid on top of the default pop-up windows. Using this 
L d2 embodiment, viewers without a set-top box capable of presenting varying pop-up 
U13 windows in accord with the present invention can still present the programming 
zSa content with static pop-up windows. Additionally, broadcasters and service can 
;*-15 present the segment of programming without variable pop-up windows if desired 
H\6 or in the event of a system malfunction, or if they lack the capability of presenting 
^1 7 the variable pop-up windows of the present invention. 

!? "18 Thus, in accordance with embodiments of the invention, the ancillary 

19 information, location of pop-up window on the screen, shape of the window, size 

20 of the window, etc. can be varied to present essentially a new viewing experience 

21 each time a viewer wishes to view a particular programming segment. 

22 Those skilled in the art will recognize that the present invention has been 

23 described in terms of exemplary embodiments based upon use of a programmed 

24 processor. However, the invention should not be so limited, since the present 

25 invention could be implemented using hardware component equivalents such as 

26 special purpose hardware and/or dedicated processors which are equivalents to 

27 the invention as described and claimed. Similarly, general purpose computers, 

28 microprocessor based computers, micro-controllers, optical computers, analog 

29 computers, dedicated processors and/or dedicated hard wired logic may be used 

30 to construct alternative equivalent embodiments of the present invention. 
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1 Those skilled in the art will appreciate that the program steps used to 

2 implement the embodiments described above can be implemented using disc 

3 storage as well as other forms of storage including Read Only Memory (ROM) 

4 devices, Random Access Memory (RAM) devices; optical storage elements, 

5 magnetic storage elements, magneto-optical storage elements, flash memory, core 

6 memory and/or other equivalent storage technologies without departing from the 

7 present invention. Such alternative storage devices should be considered 

8 equivalents. 

9 The present invention is preferably implemented using a programmed 
processor executing programming instructions that are broadly described above in 

Ml flow chart form and can be stored on an electronic storage medium. However, 

02 those skilled in the art will appreciate that the processes described above can be 

r;l3 implemented in any number of variations and in many suitable programming 

4 languages without departing from the present invention. For example, the order of 

f jl 5 certain operations carried out can often be varied, and additional operations can be 

36 added without departing from the invention. Error trapping can be added and/or 

W17 enhanced and variations can be made in user interface and information 

{118 presentation without departing from the present invention. Such variations are 

1 9 contemplated and considered equivalent. 

20 While the invention has been described in conjunction with specific 

21 embodiments, it is evident that many alternatives, modifications, permutations and 

22 variations will become apparent to those skilled in the art in light of the foregoing 

23 description. Accordingly, it is intended that the present invention embrace all such 

24 alternatives, modifications and variations as fall within the scope of the appended 

25 claims. 

26 What is claimed is: 
27 



Docket No.: SNY-P4151 



PATENT 



